Apparatus and method transitioning between driving states during navigation for highly automated vechicle

ABSTRACT

A navigation apparatus for an autonomous vehicle includes circuitry configured to receive at least one route between a start location and a destination, display the at least one route on a first screen that allows selection of a first set of routes from the at least one route, receive a plurality of characteristics corresponding to each of the at least one route. Each characteristic of the plurality of characteristics is associated with a measure and a longest block within which each characteristic can be performed continuously. The circuitry further configured to divide a route of the at least one route to generate a plurality of segments based on the plurality of characteristics, and display the first set of routes, the plurality of characteristics corresponding to the first set of routes, the measure and the longest block corresponding to the plurality of characteristics on a second screen.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/083,648 filed on Mar. 29, 2016. The entire content of eachof the foregoing application is hereby incorporated by reference.

BACKGROUND

Field of the Disclosure

This application relates generally to improvements in highly automatedor autonomous vehicles. More particularly the present disclosure relatesto applying driver preferences or historic driving performance indetermining an optimized route for highly automated or autonomousvehicles.

Description of the Related Art

In a conventional navigation system, a driver can input a destinationaddress and the navigation system determines the directions to thedestination based on the map stored or accessible to the conventionalnavigation system. The conventional navigation system is equipped with aglobal positioning sensor (GPS) to determine a current position of thevehicle. Based on the current position, the conventional navigationsystem guides the driver to the destination, using audible and visualinstructions. As more than one route is available between the currentposition and the destination, the conventional navigation system choseswhich route or routes to offer the driver. Often, the route is selectedby considering factors of importance for non-autonomous driving such astravel distance or travel time. Furthermore, some navigation systemsintegrate traffic, road event such as repair or accident relatedinformation into the directions, thereby giving the driver flexibilityto select a less congested routes.

The conventional navigations, although helpful and suitable for anon-autonomous vehicle, are not adaptive and do not account for factorsof interest for autonomous driving. For example, the conventionalnavigation system routing may not be able to account for hands free timeor sleep time, which are some of the enhanced capabilities of theautonomous vehicle over the non-autonomous vehicle, during a routedetermination and selection.

Autonomous vehicles are a next generation type of automotive vehiclewith highly increased functionalities and vehicle performance in termsof driving automation. Autonomous vehicles can not only improve drivingperformance and overall vehicle performance, but also allow a driver tosleep, take hands off the steering wheel, take eyes off the road, etc.in certain situations. To enable such increased capabilities of avehicle, several smart components including smart sensors, andcommunication with driver and other vehicles is necessary. Among theincreased capabilities an efficient routing or navigation system needsto be included, as conventional navigation systems have limitedcapabilities and pose several limitations. As such, there remains acontinuing need to provide improved navigation systems for autonomousvehicles.

SUMMARY

According to an embodiment of the present disclosure, there is provideda navigation apparatus. The navigation apparatus for an autonomousvehicle includes circuitry configured to receive, via a network, atleast one route between a start location and a destination, display theat least one route on a first screen that allows selection of a firstset of routes from the at least one route, receive, via the network, aplurality of characteristics corresponding to each of the at least oneroute, each characteristic of the plurality of characteristics isassociated with a measure and a longest block within which eachcharacteristic can be performed continuously. Further the circuitry isconfigured to divide a route of the at least one route to generate aplurality of segments based on the plurality of characteristics, a firstsegment is highlighted with a first identifier corresponding to a firstcharacteristic of the plurality of characteristics and a second segmentis highlighted with a second identifier corresponding to a secondcharacteristic of the plurality of characteristics, and display thefirst set of routes, the plurality of characteristics corresponding tothe first set of routes, the measure and the longest block correspondingto the plurality of characteristics on a second screen.

Further, according to an embodiment of the present disclosure, there isprovided a method for navigation of an autonomous vehicle. The methodincludes receiving, via a network, at least one route between a startlocation and a destination, displaying, using a processing circuitry,the at least one route on a first screen that allows selection of afirst set of routes from the at least one route, and receiving, via thenetwork, a plurality of characteristics corresponding to each of the atleast one route, each characteristic of the plurality of characteristicsis associated with a measure and a longest block within which eachcharacteristic can be performed continuously. The method furtherincludes dividing, using the processing circuitry, a route of the atleast one route to generate a plurality of segments based on theplurality of characteristics, a first segment is highlighted with afirst identifier corresponding to a first characteristic of theplurality of characteristics and a second segment is highlighted with asecond identifier corresponding to a second characteristic of theplurality of characteristics, and displaying, using the processingcircuitry, the first set of routes, the plurality of characteristicscorresponding to the first set of routes, the measure and the longestblock corresponding to the plurality of characteristics on a secondscreen.

Further, according to an embodiment of the present disclosure, there isprovided a non-transitory computer-readable medium which stores aprogram which, when executed by a computer, causes the computer toperform the method for navigation of an autonomous vehicle, as discussedabove.

The forgoing general description of the illustrative implementations andthe following detailed description thereof are merely exemplary aspectsof the teachings of this disclosure, and are not restrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate one or more embodiments and,together with the description, explain these embodiments. Theaccompanying drawings have not necessarily been drawn to scale. Anyvalues dimensions illustrated in the accompanying graphs and figures arefor illustration purposes only and may or may not represent actual orpreferred values or dimensions. Where applicable, some or all featuresmay not be illustrated to assist in the description of underlyingfeatures. In the drawings:

FIG. 1 is an interior view of an autonomous vehicle with a navigationapparatus according to an exemplary embodiment of the present.

FIG. 2A illustrates a first screen of the navigation apparatus accordingto an exemplary embodiment of the present disclosure.

FIG. 2B illustrates a second screen of the navigation apparatusaccording to an embodiment of the present disclosure.

FIG. 2C illustrates a third screen of the navigation apparatus accordingto an embodiment of the present disclosure.

FIG. 3 is a flow chart of a process performed by the navigationapparatus according to an embodiment of the present disclosure.

FIG. 4 is a flow chart of a second display process for the second screenof the navigation apparatus according to an embodiment of the presentdisclosure.

FIG. 5 is a flow chart of a third display process for the third screenof the navigation apparatus according to an embodiment of the presentdisclosure.

FIG. 6 is a detailed block diagram illustrating an exemplary user deviceaccording to certain embodiments of the present disclosure.

DETAILED DESCRIPTION

The description set forth below in connection with the appended drawingsis intended as a description of various embodiments of the disclosedsubject matter and is not necessarily intended to represent the onlyembodiment(s). In certain instances, the description includes specificdetails for the purpose of providing an understanding of the disclosedembodiment(s). However, it will be apparent to those skilled in the artthat the disclosed embodiment(s) may be practiced without those specificdetails. In some instances, well-known structures and components may beshown in block diagram form in order to avoid obscuring the concepts ofthe disclosed subject matter.

Reference throughout the specification to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with an embodiment is included inat least one embodiment of the subject matter disclosed. Thus, theappearance of the phrases “in one embodiment” or “in an embodiment” invarious places throughout the specification is not necessarily referringto the same embodiment. Further, the particular features, structures orcharacteristics may be combined in any suitable manner in one or moreembodiments. Further, it is intended that embodiments of the disclosedsubject matter cover modifications and variations thereof.

It must be noted that, as used in the specification and the appendedclaims, the singular forms “a,” “an,” and “the” include plural referentsunless the context expressly dictates otherwise. That is, unlessexpressly specified otherwise, as used herein the words “a,” “an,”“the,” and the like carry the meaning of “one or more.” Additionally, itis to be understood that terms such as “left,” “right,” and the likethat may be used herein merely describe points of reference and do notnecessarily limit embodiments of the present disclosure to anyparticular orientation or configuration. Furthermore, terms such as“first,” “second,” “third,” etc., merely identify one of a number ofportions, components, steps, operations, functions, and/or points ofreference as disclosed herein, and likewise do not necessarily limitembodiments of the present disclosure to any particular configuration ororientation.

Furthermore, the terms “approximately,” “proximate,” “minor,” andsimilar terms generally refer to ranges that include the identifiedvalue within a margin of 20%, 10% or preferably 5% in certainembodiments, and any values therebetween.

FIG. 1 illustrates a navigation apparatus for an autonomous vehicleaccording to an exemplary embodiment of the present. The autonomousvehicle includes a navigation apparatus 10 and various electronic andmechanical components. While certain aspects of the disclosure areparticularly useful in connection with specific types of vehicles, theautonomous vehicle may be any type of vehicle including, but not limitedto, cars, trucks, motorcycles, busses, boats, airplanes, helicopters,lawnmowers, recreational vehicles, amusement park vehicles, trams, golfcarts, trains, and trolleys. The autonomous vehicle can refer to a fullyautonomous vehicle, semi or partially autonomous vehicle and vehiclesequipped with various advanced driver assist system (ADAS) such asadaptive cruise control and lane departure alert. Further, theautonomous vehicle may include all of the features of a non-autonomousvehicle, for example: a steering apparatus such as steering wheel 105; anavigation apparatus 10 having a navigation display 101 with a touchscreen 102; and a driving mode selector apparatus such as a mode shifter115. The vehicle may also have various user input devices, such as amode shifter 115, the touch screen 102 or button inputs 103 foractivating or deactivating one or more autonomous driving modes and forenabling a driver (or a passenger) 120 to provide information such as anavigation destination to the navigation apparatus 10. In oneembodiment, the navigation apparatus 10 can be integrated with thedashboard of the autonomous vehicle. Alternatively, the navigationapparatus 10 can be detachably attached to a dashboard or windshield ofthe autonomous vehicle.

The autonomous vehicle can optionally include more than one display. Forexample, the vehicle may include a second display 113 for displayinginformation regarding the status of the autonomous vehicle, navigationstatus information obtained from the navigation apparatus 10, or othervehicle status information received from a computer such as anelectronic control unit (ECU) installed on the autonomous vehicle. Thesecond display 113, in the present example, displays a driving mode “D”and a speed “20” indicating that the vehicle is presently in a drivemode and is moving at 20 miles per hour. In one embodiment, when thedrive switches autonomous mode, the second display 113 can display thedriving mode as “AV” indicating the vehicle is in autonomous mode.Additional modes can include, for example, a hands free mode “HF”, aneyes-off-road mode “EOR”, autopilot, etc.

In one embodiment, the navigation apparatus 10 can also communicate withvarious components of the vehicle such as the vehicle's conventional ECU(not shown) and can send and receive information from various systems ofthe autonomous vehicle, for example, a brake pedal 107, an accelerator109, and the steering wheel 105 to control the movement, speed, etc., ofthe autonomous vehicle.

The autonomous vehicle may also be equipped with a geographic positioncomponent such as a GPS receiver to determine the device's latitude,longitude and/or altitude position, an accelerometer, a gyroscope oranother direction/speed detection devices to determine the direction andspeed of the vehicle or changes thereto. Furthermore, the vehicle mayalso include components for detecting objects and conditions external tothe vehicle such as other vehicles, obstacles in the roadway, trafficsignals, signs, trees, etc. The detection system may include lasers,sonar, radar detection units (such as those used for adaptive cruisecontrol), cameras, or any other detection devices which record data andsend signal to the ECU. Furthermore, the autonomous vehicle can beequipped with a DSRC (direct short range communication) sensor, and AV(autonomous vehicle) penetration sensor, that allow detection of otherautonomous vehicles within the range of the sensors and also enablecommunication with other autonomous vehicles. The AV penetration sensorcan include light detection and ranging (LIDAR) that provides range ordistance information and a stereo camera that allows for objectrecognition.

The aforementioned sensors can allow the vehicle to understand andpotentially respond to its environment in order to maximize safety forpassengers as well as objects or people in the environment. It will beunderstood that the vehicle types, number and type of sensors, thesensor locations, the sensor fields of view, and the sensors' sensorfields are merely exemplary. Various other configurations may also beutilized.

In one embodiment, the sensors described above can also receive inputfrom sensors installed on a non-autonomous vehicle. For example, thesesensors may include tire pressure sensors, engine temperature sensors,brake heat sensors, brake pad status sensors, tire tread sensors, fuelsensors, oil level and quality sensors, air quality sensors (fordetecting temperature, humidity, or particulates in the air),precipitation sensors for automatic windshield wipers, etc.

The navigation apparatus 10, ECU or the server 20 can receive ortransfer information to and from other computers. For example, a mapstored on the server 20 may be received or transferred from the server20 to the navigation apparatus 10, the sensor data collected from thesensors of the autonomous vehicle can be transmitted to the server 20for processing. Also, data collected from other vehicles can be storedin the database of the server 20 and the data can be used for navigationpurposes. As such, sensor information, and navigation related functions(described with reference to FIGS. 2A-2C and 3-5) can be implementedusing a central architecture, where the information from aforementionedsensors and the navigation related functions are processed by a singlecontroller such as the ECU. Alternatively, a distributed architecturecan be used, in which the navigation related functions are processedpartially by the navigation apparatus 10 and partially by the server 20.

The navigation apparatus 10 can be a human machine interface (HMI) thatallows user to view and select different routes from a start location toa destination based on certain characteristics of a route. Thecharacteristics of a route refer to various states the driver 110 canoccupy while the autonomous vehicle is in motion and other factorsspecific to the route. For example, the characteristics of the route canbe driver states such as hands free, eyes-off-road, sleeping, reclining,forward seating, manual, eyes required to be on a dedicated clonabledisplay etc., while other factors can be safety, travel time, traveldistance, etc. The notion of the clonable display refers to a dedicateddisplay onto which the driver can clone the route on an external devicesuch as a smartphone or a tablet. The external device can be placed highon the dashboard, as such the timing for reengagement of the eyes once awarning is issued can be known more precisely than a generaleyes-off-road state. So, there may be stretches of road where the eyeson the clonable display is allowed, but the eyes off the road is not.The HMI can include a software application executed on a controller ofthe navigation apparatus 10 to display more than one screen, which canbe further configured to accept user inputs and display results. Invarious embodiments of the present disclosure, three screens arediscussed by way of non-limiting examples to demonstrate various aspectsof the present disclosure.

The different characteristics of the route determined by the navigationapparatus 10 can also enable the autonomous vehicle to be operated in ateammate mode. In the teammate mode the autonomous vehicle performscertain operation such as steering, acceleration, etc., without driverinput along certain sections of the route, while in certain sectionsinvolving winding road, harsh weather, heavy traffic conditions, drivingfor pleasure, etc. the autonomous vehicle can pass the control over tothe driver.

FIG. 2A illustrates a first screen 200 of the navigation apparatus 10according to an exemplary embodiment of the present disclosure. Thefirst screen 200 of the navigation apparatus 10 displays at least oneroute button such as route buttons 201, 202, 203, and 204 correspondingto the routes R1, R2, R3, and R4, respectively. The route buttons201-204 can further display a summary of a route in the form ofparameters such as a travel time, hands free time, scenery around theroute, or an overview of a map with the route marked on the map. Theroutes R1-R4 can be determined by the server 20 that implements routingalgorithms, route optimization algorithms, or other navigationalgorithms as appropriate for the autonomous vehicle. Optionally, thefirst screen 200 can include a setting button 210 that allows a user to:select route preferences, display summary parameters in the routebuttons 201-204, store a current route, add favorite routes, displayspecial messages at particular points of interest during driving, etc.The aforementioned settings can also be taken into account by the server20 while determining the routes.

The route buttons 201-204 can be selectively activated to furtherdisplay the characteristics of each of the selected routes to assist theuser in choosing a route to navigate to the destination. In oneembodiment, the characteristics of each of the selected routes, can bedisplayed on a second screen 300 (further discussed with respect to FIG.2B) to enable comparison between different routes. Alternatively or inaddition, one or more routes (e.g., routes R1 and R4) can be selected bythe user based on the summary of characteristics of the selected routes(e.g., routes R1 and R4). In certain embodiments, a route (e.g., R1) canbe activated by double tapping the route button 201, by pressing andholding the route button 201, or other appropriate activation method.

FIG. 2B illustrates a second screen 300 of the navigation apparatus 10according to an embodiment of the present disclosure. The second screen300 displays details of the selected routes (e.g., R1, R2, and R3). Thedetails can include directions marked on a map and the characteristicsof the selected routes and measures associated with the characteristics.Further, the characteristics of the selected routes can be marked on themap to provide visual instructions to the driver 110. Thecharacteristics of the route can be, by way of non-limiting example,hands free, eyes-off-road, sleeping, reclining, forward seating, manual,safety, travel time, travel distance, scenery around the route, manualdriving for pleasure, etc.

In FIG. 2B, the characteristics of the route are presented in a tabularformat showing the attributes A1-A4 for each of the routes R1, R2 andR3. Further, each attribute A1, A2, A3 and A4 is associated with ameasure M1, M2, M3 and M4, respectively, for route R1. The measure M1(e.g., time in minutes) can be different from the measure M2 (e.g.,distance in miles). Optionally, longest blocks LB1, LB2, LB3 and LB4within a route (e.g., route R1) in which the attribute A1, A2, A3 andA4, respectively, can be performed without interruption, can bedisplayed. A longest block is a longest segment of a route, when a routeis divided into multiple route segment, along which an attribute isperformed continuously. For example, for route R1, the attribute A1 canbe the sleeping state (or hands free state) of the driver 110 measuredin time (or distance). The measure M1 corresponding to the attribute A1can be 22 mins (or 22 miles). The longest block LB1 of the sleepingstate of the driver can be 5 mins (or 5 miles), which corresponds toapproximately 22% of the measure M1. The attribute A2 can be theeyes-off-road state of the driver 110 measured in time (or distance).The measure M2 corresponding to the attribute A2 can be 20 mins (or 20miles). The longest block LB2 of the eyes-off-road state of the drivercan be 15 mins (or 15 miles). The attribute A3 can be the sleeping stateof the driver 110 measured in time (or distance). The measure M3corresponding to the attribute A3 can be 10 mins (or 10 miles). Thelongest block LB3 of the sleeping state of the driver can be 4 mins (or4 miles). The attribute A4 can be the manual state of the driver 110measured in time (or distance). The measure M4 corresponding to theattribute A4 can be 5 mins (or 5 miles). The longest block LB4 of themanual state of the driver can be 4 mins (or 4 miles). Similarly,additional attributes such as the travel time (e.g. 25 mins), the traveldistance (e.g., 20 miles), the reclining, the seat forward, etc. can bedisplayed for the route R1.

Similarly, the characteristics of route R2, by way of non-limitingexample, can be as follows. The attribute A1 (e.g., the sleeping orhands free state) can have the measure M5 (e.g., time in minutes) of 20mins with a longest block LB5 of 15 mins. The attribute A2 (e.g., theeyes-off-road state) can have the measure M6 (e.g., time in minutes) of7 mins with a longest block LB6 of 7 mins. The attribute A3 (e.g., thesleeping state) can have the measure M7 (e.g., time in minutes) of 7mins with a longest block LB7 of 7 mins. The attribute A4 (e.g., themanual state) can have the measure M8 (e.g., time in minutes) of 5 minswith a longest block LB8 of 5 mins. Additionally, travel time (e.g., 22mins), travel distance (e.g., 15 miles), reclining, seat forward, etc.can be displayed for the route R2.

Similarly, the characteristics of route R3, by way of non-limitingexample, can be as follows. The attribute A1 (e.g., the sleeping orhands free state) can have the measure M9 (e.g., time in minutes) of 7mins with a longest block LB9 of 7 mins. The attribute A2 (e.g., theeyes-off-road state) can have the measure M10 (e.g., time in minutes) of0 mins with a longest block LB10 of 0 mins. The attribute A3 (e.g., thesleeping state) can have the measure M11 (e.g., time in minutes) of 0mins with a longest block LB11 of 0 mins. The attribute A4 (e.g., themanual state) can have the measure M12 (e.g., time in minutes) of 12mins with a longest block LB12 of 12 mins. Additionally, travel time(e.g., 19 mins), travel distance (e.g., 14 miles), reclining, seatforward, etc. can be displayed for the route R3.

As such, the driver can compare the travel times, measures related toattribute A1 and the longest block related to attribute A1 (i.e.,sleeping time) for the routes R1, R2, and R3, respectively. The drivercan decide to choose route R2, since the longest block LB5 of sleepingtime is 14 mins out of 20 mins (the measure M5), although the totalsleep time (the measure M1) for the route R1 is 22 mins, which is higherthan the measure M5. The longest block LB1 of the attribute is 5 mins,since the route R1 may include multiple discontinuous segments of lessthan or equal to 5 mins, where the driver can occupy the sleeping state.On the other hand, the route R2 has the longest block LB5 of 14 mins,since the route R2 may include only two discontinuous segments of 14mins and 6 mins where the driver can occupy the sleeping state.

The characteristics of the route (e.g., the attributes A1-A4 of theroute R1) can be further used to divide the route (e.g., route R1) intomore than one route segment. A route segment can be identified as acontinuous sequence of road segments during which the driver is allowedto occupy a subset (one or more) of driving states. Further, each of theroute segments can be highlighted by identifiers such as markers basedon a color scheme. For example, the route R1 illustrates an overview ofdirections (on a map) from a start location A to a destination B andincludes a first route segment 301, a second route segment 303, and athird route segment 305. The first route segment 301 may include, forexample, three road segments on the same road or may include threedifferent roads. The first route segment 301 can be marked in greenindicating the driver 110 can occupy the sleeping (or hands free) stateonly. The second route segment 303 can be marked in red indicating thedriver 110 can occupy the manual state only. The third route segment 305can be marked in yellow indicating the driver 110 can occupy the handsfree state and the eyes-off-road state and sleeping state.

Similarly for the route R2 illustrates an overview of directions (on amap) from the start point A to the destination B and further includes afourth route segment 311, a fifth route segment 313, and a sixth routesegment 315. The fourth route segment 311 can be marked in green, thefifth route segment 313 can be marked in red, the sixth route segment315 can be marked in yellow.

Similarly for the route R3 illustrated with an overview of directions(on a map) from the start point A to the destination B and furtherincludes a seventh route segment 321, and an eighth route segment 323.The seventh route segment 321 can be marked in red indicating the driver110 can occupy the manual state only. The eighth route segment 323 canbe marked in yellow indicating the driver 110 can occupy the hands freestate and the eyes-off-road state.

Marking the route segments (e.g., the first route segment 301 and thefourth route segment 311) with identifying marks provides a visualguidance and a basis for comparison between different routes (e.g.,route R1 versus route R3) to the driver 110 to choose an appropriatepath according to his/her driving style or preference. The identifyingmarkers can also be placed next to the respective attributes A1-A4.

Certain aspects of the route can be highlighted, particularly where theautonomous vehicle has access to detailed maps, which allow theautonomous vehicle to engage the autonomous mode. For example, certainareas along the route can have detailed maps of some streets stored inthe database, while for some areas a detailed road map may not beavailable. Accordingly, the navigation apparatus 10 can highlightsections of the map (along the route) with a semi-transparent backgroundcolor, for example, autonomous areas (identified based on availabilityof a detailed map) can be marked in a faint yellow background, while anon-autonomous area can be marked in blue.

FIG. 2C illustrates a third screen 400 of the navigation apparatus 10according to an exemplary embodiment of the present disclosure. Thethird screen 400 displays real-time directions and transition messagesat transition points when the autonomous vehicle is in operation ormoving. A transition point is a location at which the driver 110 shouldtransition from one state to another. In many cases, a transition pointoccurs between two segments of a route. Thus, there is substantially thesame number of segments as there are transition points. A transitionmessage indicates a driver should occupy a particular state startingfrom the transition point. The transition message can further include anappended message, which displays characteristics of the route from afirst transition point to the next transition point. Optionally, thedetails in the appended message can be ordered, for example in anascending order or a descending order, based on the measurescorresponding to the attributes (e.g., A1-A4). The transition points,the transition messages and the appended messages can be determinedduring the route generation by the server 20, or can be generated by thecontroller of the navigation apparatus 10.

In FIG. 2C, the third screen 400 illustrates a real-time route 401(e.g., the route R1 selected by the driver 110) starting from the startlocation A, which can be the first transition point. At (or proximatelyat) the start location A, a first transition message MSG1 indicating“Disengage Autonomous Mode” or “Manual” state can be displayed. Thefirst transition message MSG1 can be appended with additional details ina first appended message MSG11. The first appended message MSG11displays the attributes A4 and A1 with the measures 10 mins and 2 mins,respectively, in descending order. The first appended message MSG1indicates the driver 110 can occupy the manual state (represented byattribute A4 in one embodiment) for 10 mins and the hands free state(represented by attribute A1 in one embodiment) for 2 mins till a secondtransition point C. At (or proximately at) the second transition pointC, a second transition message MSG2 indicating “Engage Autonomous Mode”can be displayed. Alternatively or in addition, the second transitionmessage MSG2 can indicate “Hands free”, “eyes-off-road”, etc. The secondtransition message MSG2 can be appended with additional details in asecond appended message MSG21. The second appended message MSG21displays the attributes A1, A2, A3 and A4 with the measures are 10 mins,5 mins, 5 mins, and 1 mins, respectively, in descending order. Thesecond appended message MSG21 indicates the driver 110 can occupy thehands free state (represented by the attribute A1 in one embodiment) for10 mins till a next transition point.

In one embodiment, the transition message and the appended message canbe displayed together. In another embodiment, the third screen 400 canbe configured to display the appended message upon activation of thetransition message by tapping, for example. It can be appreciated by aperson skilled in art that the configurations of the messages discussedin various embodiments are non-limiting examples and can be modifiedwithin the scope of the present disclosure.

The navigation apparatus 10 can also allow live updating of the alloweddriver states based on sensors data, real-time traffic data and suggestrerouting. The live updating can be performed for one or more routesdisplayed on the second screen 300 or the route displayed on the screen400. For instance, when the driver starts his/her commute, sleeping maybe an allowed state along a selected route, but in an event of a roadaccident such as a truck jackknife, the traffic can be rerouted. Assuch, the navigation apparatus 10 can indicate to switch to the manualmode or re-plan the routes to avoid the truck jackknife. The autonomousvehicle includes sensors capable of confirming in real-time that theexpected allowed state is correct.

The first screen 200, the second screen 300, and the third screen 400can be displayed on a same display or on multiple displays. For example,one or more of the first screen 200, the second screen 300, and thethird screen 400 can appear on the same display, may appear in asequence one after the other on the same display, on multiple screens orcombinations thereof.

FIG. 3 is a flow chart of a process performed by the navigationapparatus 10 according to an embodiment of the present disclosure. Theprocess starts when at least one route from the starting location A tothe destination B is determined and the navigation apparatus 10 receivesa signal from the server 20. In step S301, the navigation apparatus 10receives the routes (e.g., routes R1-R4) and the characteristics of theroutes (e.g., the attributes A1-A4, the measures M1-M12, and the longestblocks LB1-LB12).

In step S303, the navigation apparatus 10 displays the routes (e.g.,routes R1-R4) on the first screen 200 along with the characteristics andtransition points of each route. The user can select one or more routes(e.g., routes R1, R2, and R3) to compare the characteristics of theselected routes (e.g., routes R1, R2, and R3). The selected routes andthe characteristics of the selected routes (e.g., the attributes A1-A4,measures M1-M12, and longest blocks LB1-LB12) can be displayed on thesecond screen 300. Further, the second screen 300 includes the overviewof directions highlighted with identifiers, as discussed with referenceto FIG. 2B. After comparing the routes based on the information on thefirst screen 200 or on the second screen 300, the user can activate aroute (e.g., route R1) to navigate to the destination B. Upon activationof a route (e.g., route R1), the third screen 400 can be displayed,where the real-time directions and instructions are conveyed to theuser. The real-time directions and instructions can involve accessingand processing (via a controller) sensor data such as a global positionsensor, motion sensor, accelerometer, etc.

In step S305, a transition message to transition from a first expectedstate of the driver to a second expected state of the driver isgenerated. The transition messages generated can be, for example, thefirst transition message MSG1 indicating the first expected state“Disengage Autonomous Mode” and the second transition message MSG2 withthe second expected state “Autonomous Mode”, discussed with respect toFIG. 2C. An expected state is a state the driver is expected to occupywhen driving along a particular route segment (e.g., route segment 301in FIG. 2B). The expected state can be determined during the routedetermination by the server 20 or the navigation apparatus 10 usingspecifications of the autonomous vehicle and vehicles that will beencountered while driving along a particular route, road conditions, anurban setting, traffic conditions, information collected from the sensorof the autonomous vehicle, other information that can be used in a routedetermination. For example, a straight interstate highway or road withlow vehicle traffic can have an expected state of hands free,eyes-off-road, or sleeping. A winding road, a road with heavy traffic,or roads with several intersections can have an expected state as manualor hands free. Further, in step S305, an appended message can begenerated, as discussed with respect to FIG. 2C.

In step S307, the transition messages are updated on the third screen400 in real-time while driving along a route segment. For example, thesecond transition message MSG2 is updated on the third screen 400 whenthe autonomous vehicle is at the transition point C.

Further, in step S309, a current driver state can be monitored using thesensors installed in the autonomous vehicle. Different types of sensorsare discussed with respect to FIG. 1. For example, sensors to monitordriver state can be: a position sensor to detect the driver's handposition on the steering 105, a camera used to detect eyes-off-road,and/or a seat position sensor to detect reclining position of the seat,etc.

In step S311, the navigation apparatus 10 determines whether the currentdriver state is similar to the expected state (e.g., the manual statealong the third route segment 305) at the current location along theroute (e.g., the route R1). If so, the system keeps monitoring thedriver state, in step S309. However, if the current driver state is notsimilar to the expected state, an alert message can be generated, instep S313. The alert message may not be generated under certainconditions such as the driver 110 has his/her hands on the steering 105when the expected state is hands free. The alert message can be a visualinstruction on the third screen 400, an audio instruction, or avibration in the seat 111 or the steering 105 etc.

FIG. 4 is a flow chart of a second display process for the second screen300 performed by the navigation apparatus 10 according to an embodimentof the present disclosure. The second display process starts when theuser selects one or more routes on the first screen 200. In step S401,the routes selected by the user are highlighted with identifiersindicating the expected state of a driver along the route. For example,the route segments 301, 303, 305 of the route R1, as discussed withrespect to FIG. 2B.

In step S403, the characteristics including the travel time, the handsfree state, the eyes-off-road state, the sleeping state, the manualstate, the travel distance, and the safety of the routes are displayedbelow an overview of the map for each of the selected route R1, R2, andR3.

In step S405, road events can be identified and updated on the secondscreen 300. The road events can be an accident, repairs, bad weatherconditions, etc. The road events can be identified by the server 20 orthe controller of the navigation apparatus 10 based on data receivedfrom the sensors of the autonomous vehicle and transmitted to the secondscreen 300.

In step S407, the second screen can be updated with prompts aboutunprotected left turns. The second screen can receive unprotected leftturns information from the server 20 or the controller of the navigationapparatus 10 based on data received from the sensors of the autonomousvehicle. When the unprotected left turn is detected, the driver can bealerted to keep the hands on the wheel and the eyes on the road.

FIG. 5 is a flow chart of a third display process for the third screen400 performed by the navigation apparatus 10 according to an exemplaryembodiment of the present disclosure. The third display process startswhen the user selects a route on the first screen 200 or the secondscreen 300. In step S501, the navigation apparatus 10 starts trackingthe current position of the autonomous vehicle. The current position canbe marked on the map of the third screen 400. The current position canbe determined based on the GPS installed on the autonomous vehicle. Instep S503, the current position is compared with transition points alongthe route selected by the user. The transition points are a part of thecharacteristics of the route determined by the server 20 as discussed instep S305 of FIG. 3. Upon comparison, in step S505, a determination ismade if the current position of the autonomous vehicle is close to atransition point (e.g., the second transition point C) along the route(e.g., the real-time route 401) selected by the user. If the currentposition is not in close proximity of the transition point (e.g., thetransition point C), the third display process continuous from the stepS501.

On the other hand, if the current position is in close proximity of thetransition point (e.g., the second transition point C), a transitionmessage (e.g., the second transition message MSG2) is determined basedon the characteristics of the route at the transition point, in stepS507.

In step S509, the transition message with an expected state the driver110 should occupy is determined based on the characteristics of theroute determined by the server 20. In step S509, the transition messageis generated and transmitted to the third screen 400. The transitionmessage (e.g., the transition message MSG2) can be generated as a visualinstruction, as discussed in FIG. 2C. Alternatively or in addition, thetransition message can be in an audio form. Furthermore, an appendedmessage can be generated and transmitted to the third screen 400 asdiscussed with respect to FIG. 2C.

Further, as live updating can occur during the driving process, theallowed driver states can be updated and option to rerouting can beprompted on the third screen 400. For example, in case of a truckjackknife, the allowed state can be changed from the sleeping to themanual mode and the driver can be alerted via visual, audio, vibrationsin the steering or seat, etc.

FIG. 6 is a detailed block diagram illustrating an exemplary user device600 according to certain embodiments of the present disclosure. The userdevice 600 is an exemplary representation of the navigation apparatus10. In certain embodiments, the user device 600 may be a smartphone.However, the skilled artisan will appreciate that the features describedherein may be adapted to be implemented on other devices (e.g., alaptop, a tablet, a server, an e-reader, a camera, a navigation device,etc.). The exemplary user device 600 includes a controller 610 and awireless communication processing circuitry 602 connected to an antenna601. A speaker 604 and a microphone 605 are connected to a voiceprocessing circuitry 603.

The controller 610 executes a software application to perform thefunctions or processes discussed with respect to FIGS. 2A, 2B, 2C, 3, 4,and 5. The controller 610 may include one or more Central ProcessingUnits (CPUs), and may control each element in the user device 600 toperform functions related to communication control, audio signalprocessing, control for the audio signal processing, still and movingimage processing and control, and other kinds of signal processing. Thecontroller 610 may perform these functions by executing instructionsstored in a memory 650. For example, the processes illustrated in FIGS.3-5 may be stored in the memory 650 and executed based on the userinputs received via the first screen 200, the second screen 300, and thethird screen 400. Alternatively or in addition to the local storage ofthe memory 650, the functions may be executed using instructions storedon an external device such as the server 20 accessed on a network or ona non-transitory computer readable medium.

The memory 650 includes but is not limited to Read Only Memory (ROM),Random Access Memory (RAM), or a memory array including a combination ofvolatile and non-volatile memory units. The memory 650 may be utilizedas working memory by the controller 610 while executing the processesand algorithms of the present disclosure. Additionally, the memory 650may be used for long-term storage, e.g., of image data and informationrelated thereto. The memory 650 may be configured to store the battleview information, operation view information and list of commands.

The user device 600 includes a control line CL and data line DL asinternal communication bus lines. Control data to/from the controller610 may be transmitted through the control line CL. The data line DL maybe used for transmission of voice data, display data, etc.

The antenna 601 transmits/receives electromagnetic wave signals betweenbase stations for performing radio-based communication, such as thevarious forms of cellular telephone communication. The wirelesscommunication processing circuitry 602 controls the communicationperformed between the user device 600 and other external devices such asthe server 20 via the antenna 601. The wireless communication processingcircuitry 602 may control communication between base stations forcellular phone communication.

The speaker 604 emits an audio signal corresponding to audio datasupplied from the voice processing circuitry 603. The microphone 605detects surrounding audio and converts the detected audio into an audiosignal. The audio signal may then be output to the voice processingcircuitry 603 for further processing. The voice processing circuitry 603demodulates and/or decodes the audio data read from the memory 650 oraudio data received by the wireless communication processing circuitry602 and/or a short-distance wireless communication processing circuitry607. Additionally, the voice processing circuitry 603 may decode audiosignals obtained by the microphone 605.

The exemplary user device 600 may also include a display 620, a touchpanel 630, an operation key 640, and a short-distance communicationprocessing circuitry 607 connected to an antenna 606. The display 620may be a Liquid Crystal Display (LCD), an organic electroluminescencedisplay panel, or another display screen technology. In addition todisplaying still and moving image data, the display 620 may displayoperational inputs such as the route buttons 201-204, used for controlof the user device 600. The display 620 may additionally display a GUIhaving multiple screens as shown in FIGS. 2A-2C, for a user to controlaspects of the user device 600 and/or other devices. Further, thedisplay 620 may display characters and images (e.g., an overview of themap in FIGS. 2B and 2C) received by the user device 600 and/or stored inthe memory 650 or accessed from an external device on a network such asa camera. For example, the user device 600 may access a network such asthe Internet and display text and/or images transmitted from a Webserver.

The touch panel 630 may include a physical touch panel display screenand a touch panel driver. The touch panel 630 may include one or moretouch sensors for detecting an input operation on an operation surfaceof the touch panel display screen. The touch panel 630 also detects atouch shape and a touch area. Used herein, the phrase “touch operation”refers to an input operation performed by touching an operation surfaceof the touch panel display with an instruction object, such as a finger,thumb, or stylus-type instrument. In the case where a stylus or the likeis used in a touch operation, the stylus may include a conductivematerial at least at the tip of the stylus such that the sensorsincluded in the touch panel 630 may detect when the stylusapproaches/contacts the operation surface of the touch panel display(similar to the case in which a finger is used for the touch operation).

In certain aspects of the present disclosure, the touch panel 630 may bedisposed adjacent to the display 620 (e.g., laminated) or may be formedintegrally with the display 620. For simplicity, the present disclosureassumes the touch panel 630 is formed integrally with the display 620and therefore, examples discussed herein may describe touch operationsbeing performed on the surface of the display 620 rather than the touchpanel 630. However, the skilled artisan will appreciate that this is notlimiting.

For simplicity, the present disclosure assumes the touch panel 630 is acapacitance-type touch panel technology. However, it should beappreciated that aspects of the present disclosure may easily be appliedto other touch panel types (e.g., resistance-type touch panels) withalternate structures. In certain aspects of the present disclosure, thetouch panel 630 may include transparent electrode touch sensors arrangedin the X-Y direction on the surface of transparent sensor glass.

The touch panel driver may be included in the touch panel 630 forcontrol processing related to the touch panel 630, such as scanningcontrol. For example, the touch panel driver may scan each sensor in anelectrostatic capacitance transparent electrode pattern in theX-direction and Y-direction and detect the electrostatic capacitancevalue of each sensor to determine when a touch operation is performed.The touch panel driver may output a coordinate and correspondingelectrostatic capacitance value for each sensor. The touch panel drivermay also output a sensor identifier that may be mapped to a coordinateon the touch panel display screen. Additionally, the touch panel driverand touch panel sensors may detect when an instruction object, such as afinger is within a predetermined distance from an operation surface ofthe touch panel display screen. That is, the instruction object does notnecessarily need to directly contact the operation surface of the touchpanel display screen for touch sensors to detect the instruction objectand perform processing described herein. For example, in certainembodiments, the touch panel 630 may detect a position of a user'sfinger around an edge of the display panel 620 (e.g., gripping aprotective case that surrounds the display/touch panel). Signals may betransmitted by the touch panel driver, e.g. in response to a detectionof a touch operation, in response to a query from another element basedon timed data exchange, etc.

The touch panel 630 and the display 620 may be surrounded by aprotective casing, which may also enclose the other elements included inthe user device 600. In certain embodiments, a position of the user'sfingers on the protective casing (but not directly on the surface of thedisplay 620) may be detected by the touch panel 630 sensors.Accordingly, the controller 610 may perform display control processingdescribed herein based on the detected position of the user's fingersgripping the casing. For example, an element in an interface may bemoved to a new location within the interface (e.g., closer to one ormore of the fingers) based on the detected finger position.

Further, in certain embodiments, the controller 610 may be configured todetect which hand is holding the user device 600, based on the detectedfinger position. For example, the touch panel 630 sensors may detect aplurality of fingers on the left side of the user device 600 (e.g., onan edge of the display 620 or on the protective casing), and detect asingle finger on the right side of the user device 600. In thisexemplary scenario, the controller 610 may determine that the user iswearing the user device 600 with his/her right hand because the detectedgrip pattern corresponds to an expected pattern when the user device 600is wearing only with the right hand.

The operation key 640 may include one or more buttons (e.g., the routebutton 201-204) or similar external control elements (e.g., the buttoninputs 103 in FIG. 1), which may generate an operation signal based on adetected input by the user. In addition to outputs from the touch panel630, these operation signals may be supplied to the controller 610 forperforming related processing and control. In certain aspects of thepresent disclosure, the processing and/or functions associated withexternal buttons and the like may be performed by the controller 610 inresponse to an input operation on the touch panel 630 display screensrather than the external button, key, etc. In this way, external buttonson the user device 600 may be eliminated in lieu of performing inputsvia touch operations, thereby improving water-tightness.

The antenna 606 may transmit/receive electromagnetic wave signalsto/from other external apparatuses, and the short-distance wirelesscommunication processing circuitry 607 may control the wirelesscommunication performed between the other external apparatuses.Bluetooth, IEEE 802.11, and near-field communication (NFC) arenon-limiting examples of wireless communication protocols that may beused for inter-device communication via the short-distance wirelesscommunication processing circuitry 607.

The user device 600 may include a motion sensor 608. The motion sensor608 may detect features of motion (i.e., one or more movements) of theuser device 600. For example, the motion sensor 608 may include anaccelerometer to detect acceleration, a gyroscope to detect angularvelocity, a geomagnetic sensor to detect direction, a geo-locationsensor to detect location, etc., or a combination thereof to detectmotion of the user device 600. In certain embodiments, the motion sensor608 may generate a detection signal that includes data representing thedetected motion. For example, the motion sensor 608 may determine anumber of distinct movements in a motion (e.g., from start of the seriesof movements to the stop, within a predetermined time interval, etc.), anumber of physical shocks on the user device 600 (e.g., a jarring,hitting, etc., of the electronic device), a speed and/or acceleration ofthe motion (instantaneous and/or temporal), or other motion features.The detected motion features may be included in the generated detectionsignal. The detection signal may be transmitted, e.g., to the controller610, whereby further processing may be performed based on data includedin the detection signal. The motion sensor 608 can work in conjunctionwith a Global Positioning System (GPS) circuitry 660.

The user device 600 may include a camera circuitry 609, which includes alens and shutter for capturing photographs of the surroundings aroundthe user device 600. In an embodiment, the camera circuitry 609 capturessurroundings of an opposite side of the user device 600 from the user.The images of the captured photographs can be displayed on the displaypanel 620. A memory circuitry saves the captured photographs. The memorycircuitry may reside within the camera circuitry 609 or it may be partof the memory 650. The camera circuitry 609 can be a separate featureattached to the user device 600 or it can be a built-in camera feature.Furthermore, the camera circuitry 609 can be configured to detectfeatures of motion (i.e., one or more movements) of the user device 600.

The software application executed on the user device 600 is anapplication that requests data processing from the server 20 via awireless network. The server 20 includes a storage controller thatmanages the database on a disk and the query application manager thatexecutes SQL (structured query language) statements against this data onthe disk or the database. The query application manager 650 alsoimplements processing functions (e.g., query syntax analysis,optimization, and execution plan generation) as well as a simple networkcommunication function to send and receive signal from a networkcontroller.

In the above description, any processes, descriptions or blocks inflowcharts should be understood as representing modules, segments orportions of code which include one or more executable instructions forimplementing specific logical functions or steps in the process, andalternate implementations are included within the scope of the exemplaryembodiments of the present advancements in which functions can beexecuted out of order from that shown or discussed, includingsubstantially concurrently or in reverse order, depending upon thefunctionality involved, as would be understood by those skilled in theart.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the present disclosures. Indeed, the novel methods, apparatusesand systems described herein can be embodied in a variety of otherforms; furthermore, various omissions, substitutions and changes in theform of the methods, apparatuses and systems described herein can bemade without departing from the spirit of the present disclosures. Theaccompanying claims and their equivalents are intended to cover suchforms or modifications as would fall within the scope and spirit of thepresent disclosures. For example, this technology may be structured forcloud computing whereby a single function is shared and processed incollaboration among a plurality of apparatuses via a network.

What is claimed is:
 1. A navigation apparatus for an autonomous vehiclecomprising: circuitry configured to determine a plurality of routesbetween a start location and a destination, identify characteristics foreach of the plurality of routes, each characteristic associated with atleast one route of the plurality of routes and a corresponding measure,divide each route of the plurality of routes into segments based on thecharacteristics, determine longest blocks within each route, the longestblocks corresponding to a longest segment of the respective routes alongwhich the characteristics associated with the respective routes can beperformed continuously, said each characteristic identifying a state adriver of the autonomous vehicle can occupy while the autonomous vehicleis travelling to the destination, display the segmented plurality ofroutes, for each route the display including the characteristicsassociated with the route, the corresponding measure, and thecorresponding longest blocks, and display a real-time route selectedfrom the plurality of routes by the driver of the autonomous vehicle. 2.The navigation apparatus according to claim 1, wherein thecharacteristics include a travel time, a travel distance, a hands freestate, an eyes-off-road state, a sleeping state, a reclining state, aseat forward state, a manual state, a safety factor and a fuel economyfactor.
 3. The navigation apparatus according to claim 1, wherein theprocessing circuitry is further configured to display the plurality ofroutes including a summary of each of the routes.
 4. The navigationapparatus according to claim 3, wherein the summary is based on thecharacteristics associated with each route and the correspondingmeasure.
 5. The navigation apparatus according to claim 1, wherein afirst segment of the segments is highlighted with a first identifiercorresponding to a first characteristic of the characteristics and asecond segment of the segments is highlighted with a second identifiercorresponding to a second characteristic of the characteristics.
 6. Thenavigation apparatus according to claim 5, wherein the first identifieror the second identifier of the segments are based on a color-codingscheme to identify the first segment or the second segment.
 7. Thenavigation apparatus according to claim 6, wherein a first coloridentifies the first segment having the first characteristic and asecond color identifies the second segment having the secondcharacteristic.
 8. The navigation apparatus according to claim 1,wherein the corresponding measure is at least one of a time and adistance.
 9. The navigation apparatus according to claim 8, wherein thecircuitry is further configured to display at least one transition pointon the real-time route, wherein each transition point corresponds to astart of a segment of the segments associated with the real-time route.10. The navigation apparatus according to claim 9, wherein a number oftransition points is substantially equal to a number of segmentscorresponding to the real-time route.
 11. The navigation apparatusaccording to claim 9, wherein the circuitry is further configured todisplay a transition message at the transition point on the real-timeroute, the transition point corresponds to a start of a segment of thesegments corresponding to the real-time route.
 12. The navigationapparatus according to claim 11, wherein the transition message isassociated with an appended message, wherein the appended messageincludes the characteristics ordered based on the corresponding measureof each of the plurality of characteristics.
 13. A method for navigationof an autonomous vehicle, the method comprising: determining a pluralityof routes between a start location and a destination; identifyingcharacteristics for each of the plurality of routes, each characteristicassociated with at least one route of the routes and a correspondingmeasure; dividing, using processing circuitry, each route of theplurality of routes into segments based on the characteristics;determining, using, the processing circuitry, longest blocks within eachroute, the longest blocks corresponding to a longest segment of therespective routes along which the characteristics associated with therespective routes can be performed continuously, said eachcharacteristic identifying a state a driver of the autonomous vehiclecan occupy while the autonomous vehicle is travelling to thedestination; displaying, using the processing circuitry, the segmentedplurality of routes, for each route the display including thecharacteristics associated with the route, the corresponding measure,and the corresponding longest blocks; and displaying, using theprocessing circuitry, a real-time route selected from the plurality ofroutes by the driver of the autonomous vehicle.
 14. The method accordingto claim 13, further displaying, using the processing circuitry, atleast one transition point on the real-time route, a transition pointcorresponds to start of a segment of the segments associated with thereal-time route.
 15. The method according to claim 14, furtherdisplaying, using the processing circuitry, a transition message at thetransition point on the real-time route, the transition pointcorresponds to start of a segment of the segments corresponding to thereal-time route.
 16. The method according to claim 15, wherein thetransition message is associated with an appended message, where theappended message includes the characteristics ordered based on thecorresponding measure of each of the characteristics.
 17. The methodaccording to claim 13, wherein a first segment of the segments ishighlighted with a first identifier corresponding to a firstcharacteristics of the characteristics and a second segment the segmentsis highlighted with a second identifier corresponding to a secondcharacteristic of the characteristics.
 18. A non-transitorycomputer-readable medium storing a program which when executed by acomputer, causes the computer to perform a method for navigation of anautonomous vehicle, the method comprising: determining a plurality ofroutes between a start location and a destination; identifyingcharacteristics for each of the plurality of routes, each characteristicassociated with at least one route of the plurality of routes and acorresponding measure; dividing, using processing circuitry, each routeof the plurality of routes into segments based on the characteristics;determining, using, the processing circuitry, longest blocks within eachroute, the longest blocks corresponding to a longest segment of therespective routes along which the characteristics associated with therespective routes can be performed continuously, said eachcharacteristic identifying a state a driver of the autonomous vehiclecan occupy while the autonomous vehicle is travelling to thedestination; displaying, using the processing circuitry, the segmentedplurality of routes, for each route the display including thecharacteristics associated with the route, the corresponding measure,and the corresponding longest blocks; and displaying, using theprocessing circuitry, a real-time route selected from the plurality ofroutes by the driver of the autonomous vehicle.